Semi-Blind Spatially-Variant Deconvolution in Optical Microscopy with Local Point Spread Function Estimation By Use Of Convolutional Neural Networks

We present a semi-blind, spatially-variant deconvolution technique aimed at optical microscopy that combines a local estimation step of the point spread function (PSF) and deconvolution using a spatially variant, regularized Richardson-Lucy algorithm. To find the local PSF map in a computationally tractable way, we train a convolutional neural network to perform regression of an optical parametric model on synthetically blurred image patches. We deconvolved both synthetic and experimentally-acquired data, and achieved an improvement of image SNR of 1.00 dB on average, compared to other deconvolution algorithms.Comment: 2018/02/11: submitted to IEEE ICIP 2018 - 2018/05/04: accepted to IEEE ICIP 201

Free annotated data for deep learning in microscopy? A hitchhiker's guide

In microscopy, the time burden and cost of acquiring and annotating large datasets that many deep learning models take as a prerequisite, often appears to make these methods impractical. Can this requirement for annotated data be relaxed? Is it possible to borrow the knowledge gathered from datasets in other application fields and leverage it for microscopy? Here, we aim to provide an overview of methods that have recently emerged to successfully train learning-based methods in bio-microscopy.Comment: Accepted in Photoniques 10

DeepFocus: a Few-Shot Microscope Slide Auto-Focus using a Sample Invariant CNN-based Sharpness Function

Autofocus (AF) methods are extensively used in biomicroscopy, for example to acquire timelapses, where the imaged objects tend to drift out of focus. AD algorithms determine an optimal distance by which to move the sample back into the focal plane. Current hardware-based methods require modifying the microscope and image-based algorithms either rely on many images to converge to the sharpest position or need training data and models specific to each instrument and imaging configuration. Here we propose DeepFocus, an AF method we implemented as a Micro-Manager plugin, and characterize its Convolutional neural network-based sharpness function, which we observed to be depth co-variant and sample-invariant. Sample invariance allows our AF algorithm to converge to an optimal axial position within as few as three iterations using a model trained once for use with a wide range of optical microscopes and a single instrument-dependent calibration stack acquisition of a flat (but arbitrary) textured object. From experiments carried out both on synthetic and experimental data, we observed an average precision, given 3 measured images, of 0.30 +- 0.16 micrometers with a 10x, NA 0.3 objective. We foresee that this performance and low image number will help limit photodamage during acquisitions with light-sensitive samples.Comment: Submitted to IEEE ISBI 202

ASAP: a web-based platform for the analysis and interactive visualization of single-cell RNA-seq data

Motivation: Single-cell RNA-sequencing (scRNA-seq) allows whole transcriptome profiling of thousands of individual cells, enabling the molecular exploration of tissues at the cellular level. Such analytical capacity is of great interest to many research groups in the world, yet these groups often lack the expertise to handle complex scRNA-seq datasets. Results: We developed a fully integrated, web-based platform aimed at the complete analysis of scRNA-seq data post genome alignment: from the parsing, filtering and normalization of the input count data files, to the visual representation of the data, identification of cell clusters, differentially expressed genes (including cluster-specific marker genes), and functional gene set enrichment. This Automated Single-cell Analysis Pipeline (ASAP) combines a wide range of commonly used algorithms with sophisticated visualization tools. Compared with existing scRNA-seq analysis platforms, researchers (including those lacking computational expertise) are able to interact with the data in a straightforward fashion and in real time. Furthermore, given the overlap between scRNAseq and bulk RNA-seq analysis workflows, ASAP should conceptually be broadly applicable to any RNA-seq dataset. As a validation, we demonstrate how we can use ASAP to simply reproduce the results from a single-cell study of 91 mouse cells involving five distinct cell types

Weakly Supervised Deep Learning Methods for Biomicroscopy

Optical microscopy, an invaluable tool in biology and medicine to observe and quantify cellular function, organ development, or disease mechanisms, requires constant trade-offs between spatial, temporal, and spectral resolution, invasiveness, acquisition time, and post-processing effort. Deep learning technologies have enabled multiple applications that are transforming our day-to-day routines, including the way we approach microscopy. Yet despite the ever-increasing computational power, it is often the lack of labeled training data that is the limiting factor for wide adoption in this domain. Annotating data is often a lengthy and expensive task, since it involves tedious work, generally by skilled experts. In this thesis, I explored "weakly supervised" learning methods targeted at a variety of applications to enhance microscopy images and extract physical information from a single image. The specificity of these "weakly supervised" methods is the fact that they use very little prior information about the image in order to keep the effort to annotate training data as low as possible. Specifically, I reduced the dimensionality of the learning problem by targeting the experiment towards estimating the parameters of a spatially-variant point-spread function (PSF) model using a convolutional neural network (CNN), which does not require instrument- or object-specific calibration. Using such a model permitted to simulate realistically accurate training data that could be generalized, once the model was trained, to real microscopy images. I extensively benchmarked different network architectures, training datasets and simulation modalities towards the optimal PSF prediction performance and robustness to image degradation. Starting from the estimated PSF model parameters, I developed a variety of applications, such as a semi-blind spatially-variant deconvolution method for image deblurring and enhancement, a robust and fast microscopy auto-focus, a method for the estimation of the object surface from a single 2D image, and a method for the estimation of the object velocity in a fluid, all of them with minimal need for a priori knowledge about the optical setup

Estimating Nonplanar Flow from 2D Motion-blurred Widefield Microscopy Images via Deep Learning

Optical flow is a method aimed at predicting the movement velocity of any pixel in the image and is used in medicine and biology to estimate flow of particles in organs or organelles. However, a precise optical flow measurement requires images taken at high speed and low exposure time, which induces phototoxicity due to the increase in illumination power. We are looking here to estimate the three-dimensional movement vector field of moving out-of-plane particles using normal light conditions and a standard microscope camera. We present a method to predict, from a single textured wide-field microscopy image, the movement of out-of-plane particles using the local characteristics of the motion blur. We estimated the velocity vector field from the local estimation of the blur model parameters using an deep neural network and achieved a prediction with a regression coefficient of 0.92 between the ground truth simulated vector field and the output of the network. This method could enable microscopists to gain insights about the dynamic properties of samples without the need for high-speed cameras or high-intensity light exposure

Spatially-Variant CNN-Based Point Spread Function Estimation for Blind Deconvolution and Depth Estimation in Optical Microscopy

Optical microscopy is an essential tool in biology and medicine. Imaging thin, yet non-flat objects in a single shot (without relying on more sophisticated sectioning setups) remains challenging as the shallow depth of field that comes with high-resolution microscopes leads to unsharp image regions and makes depth localization and quantitative image interpretation difficult. Here, we present a method that improves the resolution of light microscopy images of such objects by locally estimating image distortion while jointly estimating object distance to the focal plane. Specifically, we estimate the parameters of a spatially-variant Point Spread Function (PSF) model using a Convolutional Neural Network (CNN), which does not require instrument- or object-specific calibration. Our method recovers PSF parameters from the image itself with up to a squared Pearson correlation coefficient of 0.99 in ideal conditions, while remaining robust to object rotation, illumination variations, or photon noise. When the recovered PSFs are used with a spatially-variant and regularized Richardson-Lucy (RL) deconvolution algorithm, we observed up to 2.1 dB better Signal-to-Noise Ratio (SNR) compared to other Blind Deconvolution (BD) techniques. Following microscope-specific calibration, we further demonstrate that the recovered PSF model parameters permit estimating surface depth with a precision of $\mathrm {2 \mu \text {m} }$ and over an extended range when using engineered PSFs. Our method opens up multiple possibilities for enhancing images of non-flat objects with minimal need for a priori knowledge about the optical setup

DeepFocus: a Few-shot Microscope Slide Auto-Focus using a Sample Invariant CNN-based Sharpness Function

Autofocus (AF) methods are extensively used in biomicroscopy, for example to acquire timelapses, where the imaged objects tend to drift out of focus. AF algorithms determine an optimal distance by which to move the sample back into the focal plane. Current hardware-based methods require modifying the microscope and image-based algorithms either rely on many images to converge to the sharpest position or need training data and models specific to each instrument and imaging configuration. Here we propose DeepFocus, an AF method we implemented as a Micro-Manager plugin, and characterize its Convolutional neural network-based sharpness function, which we observed to be depth co-variant and sample-invariant. Sample invariance allows our AF algorithm to converge to an optimal axial position within as few as three iterations using a model trained once for use with a wide range of optical microscopes and a single instrument-dependent calibration stack acquisition of a flat (but arbitrary) textured object. From experiments carried out both on synthetic and experimental data, we observed an average precision, given 3 measured images, of 0.30 +- 0.16 micrometers with a 10x, NA 0.3 objective. We foresee that this performance and low image number will help limit photodamage during acquisitions with light-sensitive samples

“ATLAS of Biochemistry”: A repository of all possible biochemical reactions for synthetic biology, metabolic engineering and metabolomics studies

Recent technical and analytical progress in the field of metabolomics has lead to the identification of vast amounts of new compounds in living organisms. However, the integration of these compounds into the context of known metabolism remains difficult. To address this challenge of incomplete knowledge, we propose a computational approach that identifies novel hypothetical reactions between known metabolites, integrates experimentally measured molecular structures into existing metabolic networks, and finally predicts chemical compounds that are probable to exist in metabolism. The computational framework BNICE.ch is used to exploit the known biochemistry contained in the Kyoto Encyclopedia of Genes and Genomes (KEGG). We summarize the vastly diverse functionalities of enzymatic reactions in a few hundred expert-curated reaction rules, each generalizing multiple biochemical reactions. We then apply these rules to all metabolites known to KEGG in order to create a database of all the biochemically plausible reactions between compounds reported to occur in living organism. This extrapolation of the known metabolism results in a network of more than 130’000 known and novel reactions, each connecting two or more KEGG compounds. The generated information has been organized into an online database, the “Atlas of Biochemistry”, and is available under http://lcsb-databases.epfl.ch/atlas/

ATLAS of Biochemistry: A Repository of All Possible Biochemical Reactions for Synthetic Biology and Metabolic Engineering Studies

Because the complexity of metabolism cannot be intuitively understood or analyzed, computational methods are indispensable for studying biochemistry and deepening our understanding of cellular metabolism to promote new discoveries. We used the computational framework BNICE.ch along with cheminformatic tools to assemble the whole theoretical reactome from the known metabolome through expansion of the known biochemistry presented in the Kyoto Encyclopedia of Genes and Genomes (KEGG) database. We constructed the ATLAS of Biochemistry, a database of all theoretical biochemical reactions based on known biochemical principles and compounds. ATLAS includes more than 130 000 hypothetical enzymatic reactions that KEGG connect two or more KEGG metabolites through novel enzymatic reactions that have never been reported to occur in living organisms. Moreover, ATLAS reactions integrate 42% of KEGG metabolites that are not currently present in any KEGG reaction into one or more novel enzymatic reactions. The generated repository of information is organized in a Web-based database (http://lcsb-databases.epfl.ch/atlas/) that allows the user to search for all possible routes from any substrate compound to any product. The resulting pathways involve known and novel enzymatic steps that may indicate unidentified enzymatic activities and provide potential targets for protein engineering. Our approach of introducing novel biochemistry into pathway design and associated databases will be important for synthetic biology and metabolic engineering